Message-oriented Middleware (MOM) provides an infrastructure that supports messaging between application components, including application components distributed over heterogeneous platforms. MOM is increasingly being deployed in the context of cloud computing—that is, in “the cloud”—where computing resources are delivered as services over one or more networks (e.g., the Internet).
Messaging can enable distributed communication between loosely coupled components. According to some message-based protocols, a component sends a message to a destination (e.g., a queue), and the recipient retrieves the message from the destination. However, the sender and the receiver do not have to be available at the same time in order to communicate, nor, in some cases, know anything about each other. The sender and the receiver need to know only what message format and destination to use. The messaging thus may be asynchronous, such that once a message has been sent, the sender can continue to do other work without waiting for a response. In other, implementations, the messaging may be synchronous.
The message-based protocols supported by MOM often use simple, highly flexible message formats, allowing messages to encapsulate a variety of data used by distributed components. This simplicity can lead to various implementation problems, particularly at scale. For example, Java Message Service (JMS), which is a popular MOM API for sending messages between message producers and message consumers, defines a message format having a limited amount of header data and a minimally defined body format such that JMS messages can transport a wide variety of data with relatively little overhead. The simple priority scheme supported by JMS, however, does not scale well to systems that process a large number of messages or to multi-tenant systems in which different service level objectives apply to different tenants.
The JMS message header includes a priority field (JMSPriority) so that a message producer can set a message priority for a message when placing the message in a message queue. The message priority can only be set to 0-9 by the message producer (if no priority is set, JMS defaults to 4). To facilitate processing based on priority, a JMS message consumer may implement a message selector to select messages from a message queue having a particular priority range. The JMS prioritization scheme can be considered “consumer-based” because it is the consumers that control when messages are processed from the queue.
The JMS priority levels are relative priorities. For example, level 9 messages in a queue will have priority over level 8 messages regardless of how long the level 8 messages have been in the queue. Additionally, the JMS consumer cannot select from the JMS message queue based on age because the JMS message selector provides no guarantee of ordering. The use of relative priorities and lack of ordering can lead to a situation in which a glut of higher priority messages cause lower priority messages to languish in the message queue. This can result in a component failing to meet service level objectives with respect to the lower priority messages, even if the application could have waited to process some of the higher priority messages until later while still meeting the service level objectives for the higher priority messages.
One solution to the problems discussed above is to run multiple JMS consumers, each implementing a selector for a particular limited range of priorities. This solution, however, can result in idle threads—that is, wasted processing resources—when there are no messages in the message queue in the priority ranges for particular consumers. On the other hand, if there are many messages in the queue in the priority range for a small number of consumers, the limited number of threads allocated to process the messages in the priority range can lead to a processing bottle neck, starving downstream processes of tasks.